Evaluation

UNDP defines evaluation as “a rigorous and independent assessment of either completed or ongoing activities to determine the extent to which they are achieving stated objectives and contributing to decision making” . Evaluations involve systematic and objective assessments of an on-going or completed programme, including the design, implementation and results with the aim to determine its effectiveness, impact and sustainability. Evaluations rely on the baselines and information provided through monitoring and involve more extensive analysis to understand the outcomes and impacts of a scheme/project. The insights from evaluations are also useful when planning and designing new interventions

Monitoring and Evaluation are both key processes in policy making by linking interventions with their outcomes/impacts and ensuring. Both these processes focus on improving performance by identifying outputs, outcomes and impact, setting goals for each of them and measuring the achievements against the defined goals. This information can provide insights into whether a scheme/project is on track in the right direction, how the various elements of a scheme/project are working, and which of these need to be re-considered or modified.

Our Work

DMEO conducts independent evaluation studies of both Central Sector (CS) and Centrally Sponsored Schemes (CSS) in addition to fostering institutional and knowledge capacity in India for evaluations through engagement with Union and State Ministries and Departments, establishing multidimensional gateways for dissemination of knowledge and evidence and strengthening the role of evaluation through outcome budgeting.

Centrally Sponsored Schemes Evaluations

To ensure efficient and effective use of public resources, the Government of India has made evaluation of the Centrally Sponsored Schemes (CSS) and Central Sector (CS) schemes mandatory before the schemes come up for fresh appraisal before the start of the 15th Finance Commission. The Government has assigned DMEO, NITI Aayog the responsibility to conduct independent third-party evaluation of all CSS in a time-bound manner so that the findings of the evaluation are made available to appropriate authorities for determining the rationalization of the schemes. In 2019, the evaluation of the CSS under the 28 Umbrella CSS was undertaken by DMEO through 10 sector evaluation studies. The CSS schemes are categorised into ten packages as given below:

 

 

The schemes were assessed based on the internationally recognized evaluation framework RCEESI (Relevance, Coherence, Efficiency, Effectiveness, Sustainability and Impact) and were contextualized to include equity (RCEESI+E) in line with our national priorities. The reports have been shared with the Department of Expenditure and respective line Ministries/Departments for consideration. Compendiums of Best Practices emerging from the studies have been shared with Chief Secretaries of States, and can also be accessed here: https://dmeo.gov.in/evaluation/dmeo-evaluation-studies

Flagship Evaluation Studies

DMEO undertakes evaluation of selected programmes on the request of the implementing Ministries/ Departments of the Government of India. DMEO also conducts high-impact flagship evaluations aimed at providing timely actionable insights to decision-makers, employing cutting-edge evaluation tools, methodologies, and technologies. Technical assistance is also provided to various Ministries and Departments on request, including for developing terms of reference, navigating the procurement process etc.

Quick Assessment Studies

DMEO also undertakes quick assessment studies from time to time. A quick assessment study is a short duration, need-based systematic and objective study with the goal to assess government schemes and programmes in line with standard international evaluation principles. It suggests timely remedial action, correct deficiencies and delivery of schemes and programs in the most effective and efficient way. The findings from such studies may be used to drive reform and future policy initiatives across the Ministries & Departments. These studies are important as conventional methods of evaluation may not be suitable to provide urgent policy inputs and these studies help in quickly assessing performance of schemes as well as shortfalls on achievements and makes them amenable to timely course correction.

DMEO has developed a toolkit for conducting Quick Assessment Studies to introduce the concept of rapid appraisal processes by providing a set of concise, practical and user friendly tools. The guidelines will help evaluation offices to design and execute quick and short studies in line with standard international evaluation principles. It can be an effective tool for rapid assessment of recently launched schemes/and or policies. (Link to toolkit)

Research and Analysis Work

DMEO also utilises its core strength of in-house professionals from different backgrounds with varying research interests to undertake intensive research on critical development issues under its Research and Analysis Work (RAW) Initiative. Under this initiative, research proposals by the technical team are discussed every week on the relevance, feasibility and research methods of the study. The RAW initiative also includes a weekly presentation of academic research conducted by the technical team members within DMEO and/or external experts who are invited to present their research on a weekly basis.

Other evaluations

DMEO undertakes evaluation of selected programmes, either suo-motu or on the request of the implementing Ministries/ Departments of the Government of India. The purpose of programme evaluations include an objective assessment of the longer term impacts of the development programmes, identification of the areas and reasons for the successes and failures at different stages of programme execution; suggesting mid-course corrections and disseminating lessons for the future. The parameters for evaluation would include the relevance, effectiveness, efficiency, equity and sustainability of service delivery, outcomes and impacts.
 

 DMEO also conducts high-impact flagship evaluations aimed at providing timely actionable insights to decision-makers, building evaluation capacity in the M&E ecosystem in India, and piloting cutting-edge evaluation tools, methodologies, and technologies.
 

 Technical assistance is also provided to various Ministries and Departments on request, including for developing terms of reference, navigating the procurement process etc.

Other Activities

DMEO is working with various partners to leverage and activate existing capacities in the external ecosystem to undertake a greater depth and breadth of activities than DMEO can undertake by itself. DMEO aims to explore multi-stakeholder engagements involving organizations like funding partners, technical partners and researchers, technology and data partners, field partners etc. DMEO will act as a conduit for M&E knowledge and skills: learning from leading organizations in India and abroad and helping channel this learning towards Union Ministries, States and other external organizations, thereby raising the equilibrium of the entire M&E ecosystem.
 

The Office also actively participates in NITI-wide initiatives and provides support as and when requested.

 

i) Development Evaluation Advisory Committee (DEAC)

The Development Evaluation Advisory Committee (DEAC) was reconstituted under the chairmanship of the Hon’ble Vice Chairman, NITI Aayog on 30.06.2021 as an apex Committee to guide the Development Monitoring and Evaluation Office to lead the way for institutionalizing evaluation of government schemes and other initiatives, and promote evaluation culture and ecosystem, expertise and partnerships in the country. This Committee guides the formulation and implementation of the evaluation plan of DMEO in the medium and long term. It also guides DMEO on the issues of engagement with the Union Ministries and States for capacity building for evaluation in addition to strengthening the role of evaluation in outcome budgeting.

 

ii) Expert Consultations

DMEO also invites evaluation experts and top class institutions from time to time for capacity building and dissemination of learnings. It also organizes and participates in conferences and webinars for exchanging and disseminating knowledge.

The first consultative meet was held on 15th July 2021, which deliberated on the challenges posed by COVID-19 in conducting research through field survey and the possible solutions/alternatives to the same. It also shed light into the different challenges in outsourcing survey work in India and ways to ensure quality and reliability in such engagements. (Link)

The second consultative meeting was held on 24th August 2021, in which nutritional and food security aspects in India under the legal framework of National Food Security Act, 2013 were discussed. The participants also shared their views on the survey design, instruments and partnership with institutions at local level such as regional universities and civil society organizations.